New `repeated_measures_d()` #618

mattansb · 2023-11-06T23:20:06Z

This PR introduces the new repeated_measures_d() function, thanks to the help of @bwiernik and @MatthewBJane (#372).

It can be installed via:

remotes::install_github("easystats/effectsize/@paired-d-2")

The function is fully documented (but @bwiernik and @MatthewBJane feel free to suggest edits), with an addition to existing SMD vignette, and effectsize(<t.test(paired = TRUE)>) support.

Waiting to hear from Jeff Rouder if we can use his simulated puzzle data as our dataset. If I won't hear from him in a couple of days, I'll just simulate my own.

Demo of new functionality:

library(effectsize)

# Paired data -------

data("sleep")
sleep2 <- reshape(sleep, direction = "wide",
                  idvar = "ID", timevar = "group")

repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2)
#> d (rm) |         95% CI
#> -----------------------
#> -0.75  | [-1.17, -0.33]
#> 
#> - Adjusted for small sample bias.

# Same as:
# repeated_measures_d(sleep$extra[sleep$group==1],
#                     sleep$extra[sleep$group==2])
# repeated_measures_d(extra ~ group | ID, data = sleep)


# More options:
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, mu = -1)
#> d (rm) |        95% CI
#> ----------------------
#> -0.28  | [-0.65, 0.09]
#> 
#> - Adjusted for small sample bias.
#> - Deviation from a difference of -1.
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, alternative = "less")
#> d (rm) |        95% CI
#> ----------------------
#> -0.75  | [-Inf, -0.40]
#> 
#> - Adjusted for small sample bias.
#> - One-sided CIs: lower bound fixed at [-Inf].

# Other methods
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "av")
#> d (av) |         95% CI
#> -----------------------
#> -0.76  | [-1.13, -0.39]
#> 
#> - Adjusted for small sample bias.
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "b")
#> Becker's d |         95% CI
#> ---------------------------
#> -0.72      | [-1.20, -0.24]
#> 
#> - Adjusted for small sample bias.
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "d")
#> Cohen's d |         95% CI
#> --------------------------
#> -0.80     | [-1.30, -0.30]
#> 
#> - Adjusted for small sample bias.
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "z", adjust = FALSE)
#> d (z) |         95% CI
#> ----------------------
#> -1.28 | [-2.12, -0.45]

# d_z is the same as Cohen's d for one sample (of individual difference):
cohens_d(extra.1 - extra.2 ~ 1, data = sleep2)
#> Cohen's d |         95% CI
#> --------------------------
#> -1.28     | [-2.12, -0.41]



# Repetition data -----------

data("rouder2016")

# For rm, ad, z, b, data is aggregated
repeated_measures_d(rt ~ cond | id, data = rouder2016)
#> The rm standardized difference requires paired data,
#>   but data contains more than one observation per design cell.
#>   Aggregating data using `mean()`.
#> d (rm) |         95% CI
#> -----------------------
#> -0.80  | [-1.06, -0.53]
#> 
#> - Adjusted for small sample bias.

# same as:
rouder2016_wide <- tapply(rouder2016[["rt"]], rouder2016[1:2], mean)
repeated_measures_d(rouder2016_wide[, 1], rouder2016_wide[, 2])
#> d (rm) |         95% CI
#> -----------------------
#> -0.80  | [-1.06, -0.53]
#> 
#> - Adjusted for small sample bias.

# For r or d, data is not aggragated:
repeated_measures_d(rt ~ cond | id, data = rouder2016, method = "r")
#> d (r) |         95% CI
#> ----------------------
#> -0.26 | [-0.33, -0.18]
#> 
#> - Adjusted for small sample bias.
repeated_measures_d(rt ~ cond | id, data = rouder2016, method = "d", adjust = FALSE)
#> Cohen's d |         95% CI
#> --------------------------
#> -0.25     | [-0.32, -0.18]

# d is the same as Cohen's d for two independent groups:
cohens_d(rt ~ cond, data = rouder2016, ci = NULL)
#> Cohen's d
#> ---------
#> -0.25    
#> 
#> - Estimated using pooled SD.


# From t.test -----------------------------------------------

Tt <- t.test(sleep2$extra.1, sleep2$extra.2, paired = TRUE)

effectsize(Tt, type = "rm_b")
#> Becker's d |         95% CI
#> ---------------------------
#> -0.72      | [-1.20, -0.24]
#> 
#> - Adjusted for small sample bias.

^{Created on 2023-11-07 with reprex v2.0.2}

#372

also rename type -> method

mattansb · 2023-11-06T23:23:06Z

@bwiernik I also added some unicode subscript letters just for you ;)

library(effectsize)

data("sleep")
sleep2 <- reshape(sleep, direction = "wide",
                  idvar = "ID", timevar = "group")

repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "av")
#> d (av) |         95% CI
#> -----------------------
#> -0.76  | [-1.13, -0.39]
#> 
#> - Adjusted for small sample bias.

options(es.use_symbols = TRUE)
repeated_measures_d(Pair(extra.1, extra.2) ~ 1, data = sleep2, method = "av")
#> dₐᵥ   |         95% CI
#> ----------------------
#> -0.76 | [-1.13, -0.39]
#> 
#> - Adjusted for small sample bias.

codecov · 2023-11-06T23:34:20Z

Codecov Report

Merging #618 (bfd67c6) into main (448e39b) will increase coverage by 0.18%.
The diff coverage is 94.03%.

❗ Current head bfd67c6 differs from pull request most recent head 6e9f42c. Consider uploading reports for the commit 6e9f42c to get more accurate results

@@            Coverage Diff             @@
##             main     #618      +/-   ##
==========================================
+ Coverage   90.55%   90.74%   +0.18%     
==========================================
  Files          56       57       +1     
  Lines        3419     3564     +145     
==========================================
+ Hits         3096     3234     +138     
- Misses        323      330       +7

Files	Coverage Δ
R/cohens_d.R	`96.03% <100.00%> (+0.16%)`	⬆️
R/effectsize.R	`96.77% <ø> (ø)`
R/effectsize.htest.R	`91.94% <100.00%> (+0.15%)`	⬆️
R/print.effectsize_table.R	`97.39% <100.00%> (+0.02%)`	⬆️
R/rank_diff.R	`81.15% <ø> (ø)`
R/utils_ci.R	`97.01% <ø> (ø)`
R/repeated_measures_d.R	`98.78% <98.78%> (ø)`
R/common_language.R	`94.71% <33.33%> (-0.90%)`	⬇️
R/utils_validate_input_data.R	`85.26% <88.46%> (+1.93%)`	⬆️

R/repeated_measures_d.R

MatthewBJane · 2023-11-07T01:11:58Z

R/repeated_measures_d.R

+
+  d <- (m - mu) / s
+  se <- sqrt(
+    (df / (df - 2)) * (m_V / (s^2)) +


so I think this (df/(df-2)) term is a small sample correction factor that seems to be applied broadly (i have seen it in other SE equations too). It probably does not matter all that much, but it might be good to stay consistent

Yeah, but I don't feel comfortable making that change myself 😅😳

[skip]

mattansb · 2023-11-07T07:20:24Z

Alright thanks @MatthewBJane !

mattansb added 21 commits November 1, 2023 13:01

init

87e89a1

#372

#372

cabc7c8

clean up and start docs

20871ed

Update _pkgdown.yml

b6a1c5d

add rouder2016

586ab09

add examples

517368b

more docs

67d7567

Update repeated_measures_d.R

3293a7b

hunting down references

31145a5

Update repeated_measures_d.R

b8bb32d

complete docs and names

21402c1

also rename type -> method

add message about agging

7a59f43

Update NEWS.md

d9a4235

fix footnote bug

7d80dc9

Update utils_validate_input_data.R

bc0418d

tests

a159bf8

clean up

67ae289

vignette

2f5e89c

effectsize for htest

534a8f2

remove old paired d

a0a2053

better SE for d_av

1d857b5

mattansb mentioned this pull request Nov 6, 2023

Improved d values for paired designs #372

Closed

mattansb added 2 commits November 7, 2023 01:25

styler

5c726a3

version bump

81db88b

mattansb and others added 3 commits November 7, 2023 01:37

fix CI errors

716a95d

Merge branch 'main' into paired-d-2

034955a

Update repeated_measures_d.R

ce37ae0

MatthewBJane reviewed Nov 7, 2023

View reviewed changes

mattansb added 3 commits November 7, 2023 08:32

use exact formula from Pustejovsky et al 2014

6a4cef7

[skip]

clean up

5eb5ea0

[skip]

notes

6e9f42c

[skip]

mattansb merged commit b0caa91 into main Nov 7, 2023
23 of 26 checks passed

mattansb deleted the paired-d-2 branch November 7, 2023 07:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New `repeated_measures_d()` #618

New `repeated_measures_d()` #618

mattansb commented Nov 6, 2023 •

edited

Loading

mattansb commented Nov 6, 2023

codecov bot commented Nov 6, 2023 •

edited

Loading

MatthewBJane Nov 7, 2023

mattansb Nov 7, 2023

mattansb commented Nov 7, 2023

New repeated_measures_d() #618

New repeated_measures_d() #618

Conversation

mattansb commented Nov 6, 2023 • edited Loading

mattansb commented Nov 6, 2023

codecov bot commented Nov 6, 2023 • edited Loading

Codecov Report

MatthewBJane Nov 7, 2023

Choose a reason for hiding this comment

mattansb Nov 7, 2023

Choose a reason for hiding this comment

mattansb commented Nov 7, 2023

New `repeated_measures_d()` #618

New `repeated_measures_d()` #618

mattansb commented Nov 6, 2023 •

edited

Loading

codecov bot commented Nov 6, 2023 •

edited

Loading